# Low VRAM Inference

This is an unofficial GPTQ-Int3 quantized version based on the Qwen2.5-VL-7B-Instruct model, suitable for multimodal image-text-to-text tasks.

This is the GPTQ-Int4 quantized version of the Qwen2.5-VL-3B-Instruct model, suitable for multimodal tasks involving image-to-text and text-to-text, supporting both Chinese and English.

A lightweight multimodal model designed for analyzing video content, capable of processing video, image, and text inputs to generate text outputs.

SmolVLM2-256M-Video is a lightweight multimodal model specifically designed for analyzing video content, capable of processing video, image, and text inputs to generate text outputs.

The 4Bit quantized version of Molmo-7B-D-0924, which reduces VRAM usage through the NF4 quantization strategy and is suitable for environments with limited VRAM.

INT4 quantized version of Llama 3.1 8B Instruct, quantized using AutoAWQ tool, suitable for multilingual dialogue scenarios.

RWKV-4 is a large language model combining the strengths of RNN and Transformer, featuring high performance, fast inference, and efficient training

MOSS is an open-source conversational language model supporting plugin enhancement, with 16 billion parameters, capable of Chinese-English dialogue and tool calling.